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ABSTRACT 


A modified binary search routine was developed by 
Tysver to provide inputs (estimates of parameters of thres- 
hold stimulus distributions) to the Probit and Staircase 
Pacmaliguies tor Sensivivicy Cesting. A high speed digital 
eepuver Was used to provide simulated sensitivity data 
in order to test the modified search (more specifically, 
study the parameter estimates generated by the search). 

The results show that the modified search does not 
give practical input information to the techniques mentioned 
above. However, further refinements of extensions to the 
basic search patterns should yield better estimates of the 


eisvuribution parameters. 
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Pa een ODUC TION 


The estimations of parameters describing distributions 
of outcomes in various physical situations are usually 
easy to compute and have known accuracy. The best estimate 
Sieuie mean time to failure of a light bulb, for example, 
isasamply the average failure time of a sample of light 
eo ae west eestamates Of the mean and variance of a 
normally distributed outcome are the sample average and 
sample variance using well known formulas. The samples 
Pim tcoc Casco ace made up Of a collection of point esti- 
mates of the population mean. 

However, there exists a class of physical situations 
maemauure LOr Which the parameters of the distributions 
cannot be estimated by the usual point estimate techniques. 
Examples are the threshold stimulus to detonate an explosive 
eiaorecS une amount Ol insecticide necessary Co kill a pesky 
mosquito, and the intensity of a light source necessary 
for visual perception at a given range. These examples 
share a common characteristic: they all deal with an appli- 
Savion of some form of impulse stimulus. The study of this 
class of physical situations has been labeled "sensitivity 
enalysis." 

When applying a stimulus to a certain sense, one of two 
mespolses Can occur. The subject has a positive response 


(e.g., detonates, dies, sees the light source) or a 





negative response (e.g., fails to detonate, lives, fails to 
see the light source). The response is simply a Bernoulli 
random variabie and the probability of a response varies 
with the stimulus level applied. 

Major ciroruc in the pact Mave been devoted to statis— 
tical techniques centered on the density function and point 
ecmimaves. However, estimates of parameters of sensitivity 
distributions cannot be established by usual point estima- 
Pon methods Since data realized in sensitivity testing 
PmmOuvaimca tron dicGrinpuvaen LTunections, not density 
fume t1 ons . 

A few techniques have been developed over the years 
memaccomplish the task of providing parameter estimates 
mom Sermisitivivtvy data. These include the Probit technique, 
as described by Finney [2] . the Staircase or Bruceton method 
reviewed by Dixon and Mood [1], the Countback method of 
Lewis [4], and others. Many methods are devoted to finding 
the 50% level (the level at which one would expect 50% 
positive responses to the stimulus (4) ). Others claim to 
estimate parameters of sensitivity distributions accurately 
but require knowledge or assumptions about the parameters 
peor to testing. 

The Modified Binary Search (MBS) is a method designed 
by Tysver [5] to examine physical situations for which sensi- 
tivity testing is applicable. Various levels of stimuli 
are used as a basis for computing estimates of parameters 


eevee appropriate threshold stimulus distribution. 





Tysver proposed that the estimates from MBS be used as 
inputs to previously mentioned tests (e.g. Probit, Stair- 
case) and also in predictions for safety and reliability. 

The results of [5] were examined and tested through 
Che use of simulation conducted on a high-speed electronic 
. computer. Standard Monte Carlo techniques were used to 


memerate the required sensitivity data. 





Ii. THE MODIFIED BINARY SEARCH 


A. UNDERLYING SENSITIVITY MODEL 
A brief description of the model used in the Modified 


Binary Search (MBS) is as follows. Let x be an applied 


stimulus level (xe(0,°)). Define Y f(x) to be a Bernoulli 


random variable with realizations y i Tonwa, POSSI tive 
response and y = 0 for no response. Next, define a response 


function p(x) where 
p(x) = Prob(Y = 1|x), 


and assume that p(0) = 0 and p(~) = 1. An investigator, 
mewever shoudd be able to determine a shorter interval 

a,b) with 0 < a < x < b < ~ where p(a) = 0 and p({b) = 1 
Mtia a high depree of confidence. The response function 


me, ecraphed in Figure 1. 


p(x) 





Figure 1. The Response Function 





iamerher words. p(x) is the cumulative distribution func- 
tion for a random variable X (the threshold stimulus), 


where 
Pie m= Prong ax) 4 


imemtie applied Stimulus level is greater than the thres- 
HolLawstimuus, XA, then the response will be y = l. 


ities = X. then the response willbe y = 0. “Nove prat 


Oe) 


Prob(Y 


Prob(X x ) 


1A 


1|x) 


and 


Prob(yY Ose) Prob(X > x) 1 - p(x). 


The threshold stimulus, X, or an appropriate transforma- 
tion of the same, is assumed to be normally distributed. 
The validity of this assumption does not appear to be 
@ritical in the vicinity of the 50% response level, especial- 
ly when the investigation is limited to small samples [6] . 
VitRerine Sensitivity model detined, the investigator 
is now faced with the task of preselecting the applied 
Stimulus levels (x1, X2,---, xX,,) or determining a precedure 
to choose each stimulus level Xs 4] based pen previously 
tested stimulus levels (x1, X2,..., x, ) and the observed 
mesititS (¥1, Yo,---, y,). ‘Once a sampling procedure 
Mes been selected, he conducts the investigation. The 
Bample data (yi, yo,--+; ve) he obtains contain information 
Ome vhe distribution function of the threshold stimulus. 
Rach response (y,) Pel iJcomtneeinvestigavor Whether or nov 


the random threshold stimulus (X, ) was greater than the 


2 





applied stimulus level (x,). The investigator is thus 
faced with estimating parameters of the threshold stimulus' 
Geénsity function from data describing the distribution 


humction. 


Bee DESCRIPTION OF MBS 

The investigator's first task is to choose reasonable 
Srapoates "ay ana hwrever=rnie=rangze™orrstimulus wWevels where 
p(a) =~ 0 and p(b) =~ 1. Sampling outside the interval 
(a, b) will be unnecessary if the endpoints are ehosen 
properly. For his first trial he selects x, = (a + b)/2. 
If y = 1, then he selects x, = (a + b)/4 for his next 
trial. If y = 0, however, he selects x, = 3(a + b)/4. 
The complete search procedure is diagramed in Figures 2 
through 5. If it appears that the search is converging 
MemeiecmO: sole Encdpoinus, then the investigator should 
mbesoloOn his initial choice of that endpoint. 

Meieouset= scmslvivity analysis techniques, MBS does 
meu verminate until an “inversion” occurs (Ys = 1 and 
Ys = O for some i, j where xX. < x). Six equidistant 
mevels are tested in the vicinity of the inversion. 


Let the lowest of the six levels be denoted by x, and the 


L 


highest by x Define the interval (x x) as the "pri- 


ee 
mary region" and the distance separating any two adjacent 
levels in this primary region as "DELX." Any levels tested 
outside of the primary region are separated by at least 


DELX. The preliminary search ends with one of three pos- 


Soleus ecqucibaal Outcomes, S1, S2, or S3. The number of 
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trials necessary to ee a sequence may vary with each 
experiment, but the primary region will consist of only 
three possible outcomes as depicted in Figure 6. All 
levels tested below the primary region yielded "0" respon- 


ses while those tested above the region yielded positive 


responses. 

y= 0 Ome eeilie) ni: 1 alt 
5): 

Xx =a Lm ex. xX b 

y = 0 Omen ele ae 
S2: 

Sect ee ce OX UX XK ON we GR. Cw «CO 

Vacs casa | OedaO 2 «6 6 ll 
3: 

cl ee me OX. KX. Sw sl LD 


Figure 6. Primary Region 


Once a terminal sequence (S,, S2, or S3) has been 
obtained, maximum likelihood estimates of the parameters 
en the stimulus distribution are obtained. 

Let n be the number of trials in a terminal sequence. 


the estimates, u and G6, are found by maximizing: 


n 
Il Prob(Y, 
i=] 


Y, [X53 Hd) Bouatvon scl) 


where 


N(x, 5 io) i) Wg dl 


Prob(y, = y. 


aw Aw 
i | *43 Usd) a 


Th Ga Re) SEN way = 


IES 





and 


_ x)? 


wn wn ~ 2 
N(x; 1,0) = Prob(X < x; u,0) = z é 2 a 


YONG 
oo 


ad 





In other words, given a sequence of responses (yl, 
ey es + Wed nO reece MemMeltS, she vedsse-(Xy hex 54 6 ts 5 x): find 
the maximum likelihood estimates (mle's) of the parameters 
maae Gescribe the normal distribution from which the data 
was realized. These estimates, i and 0d, can then be used 
pom npuGs Go the Stairease or Probit techniques. If more 
trials are desired in order to yield more accurate esti- 
Maves, scien the investigator should follow the extension 


procedures as described below. 


ea EBATENSION OF SEARCH 
ie exberision of 3) 

For the terminal sequence S,; the responses are 
facrily symmetric about u, SOmraal additional trials snould 
Be carried out in the primary region. Figure 7 depicts 
the stimulus levels to test in the event three or five 


additional trials are desired. 


Five Trials 


Figure fee Pec iouommOr 34 
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Pet erems lon or S52: 0r 53 


It appears that one of the ends of the primary 


weet toutairay well pinned down while the other is not. 


Therefore additional trials are 


Figure 8. 


Xx Xx Xx 


recommended as shown in 


S2 Extension 


0 0 1 0 a 


S3 Extension 


Figure 8. Extensions of S2 and S;3 


D. EXAMPLE OF A SEARCH USING MBS 


A sample experiment may go as follows. Suppose an 


mavestigator is testing for the 
This type of an event certainly 
Moememe Sensitivity class. From 
chooses a = 0 C and b = 1000 C. 
aueted at 500 avail INS Cl aides! 


The next level (temperature) at 


flashpoint ol van explosives 
Guat tres ras belonging 
past experience he roughly 
His first trial is con- 

je gvehe eleven (cio) lio etal iqey Ss abyerigishin(sis; - 


Viehtwuwemveost. the Cxplosive 


mee chus O50) 1 - Continuing his experiment he arrives at 


merminal sequence Si after eight trials. 


Trial Lewel (2) Outcome 

1 500 lene ast Lon) 

2 250 0 (no ignition) 
3 150 ines *) 

4 SHS 0 


ALi 





Trial Level (°C) Outcome 


5 625 ICS?) 

437.5 0 
7 562.5 O (inversion) 
8 MS 1 


An ordered listing of the trial levels and respective 
outcomes is as follows. The equations for the mle's will 


appear later. 


y = 0 0 pO ac. te 1. ol 1 


| 
(o) 


x 250 375 | 500 | 625 | 750 TO OOMG Te) 
Neel Saye) Sy lepoiien S 
The maximum likelihood estimates are: u = 531.25 Cc; 


ge= 81.25°C. 
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IIIT. NATURE OF PROBLEM 


In order to adequately test the MBS a digital computer 
was used for both simulation and data processing. It 
appeared that a standard analytical method (e.g. using 
derivatives) to locate the maximum of Equation (1) was 
not feasible. However, a favorable property of Equation (1) 
imome@ateude fUmeciOn 15 UNamodal over the entire range of 
i and o. Thus, search routines (e.g. Direct, Hookes-Jeeves) 
can be used to find estimates of the maximum of Equation (1) 
and the associated p and 6 as well. 

MeL Stamaves sol Giresnola parameters calculated in 
[5] were done by hand, or desk calculator, thus permitting 
MiemoesscitoTlyty of Sipnaficant round off error. The only 
inputs (response data). to Equation (1) were taken from the 
miermary inversion région, while responses outside this 
region were ignored. This seemed to have the greatest 
effect of the sequences S» and $3. Inclusion of all res- 
ponses was considered reasonable and thus written into 
meme computer program used in this study. 

In an earlier study on MBS Hicks [3] claimed that 6 
Emaerestimated o significantly with no predictable bias. 

The distribution of 6 was therefore miviestapated ian this 
study for each of the terminal sequences. In addition, 


the distribution of i was investigated. 
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IV. SIMULATION 


A. DESCRIPTION 

The computer program was written in the Fortran IV 
programming language and executed on an IBM 360/67 computer. 
Ene program wWwaS written in two sections, I and Ii. 

Scevlon = conducted sa Ssimulaved sensitivity experiment 
leading to a terminal sequence (S,, 8,, or S,;). A standard 
mompmal distribution was used as the source of the sensiti- 
vity data. The distribution endpoints (a, b) were varied 
in order to test MBS over a wide range of starting points. 
In most cases a~U(-9, -3) and b~U(3, 9). At each stimulus 
ree 1. X,» 4 random number was drawn from the simulated 
Gistribution and tested. If the random number was greater 
than Xs then Vas O. The program followed the complete 
search procedure as illustrated in Figures 2 through 5. 

A simple flow chart is diagrammed in Figure 9. 

SccCumonebiarook the Vectors (X)). Xo5e.64 x) and 
ats You ++ ve from Section I and calculated estimates 
Suet and 6 to within .03 DELX of the values y* and o* 
that maximize Equation (1). Initially a "direct" search 
was employed to locate the mle's for a given sequence. 
Different guesses of ff and G were used as inputs to Equa- 
tion (1). This method proved to be very inefficient. 


A Hookes-Jeeves search routine for two variables was tried 


20 
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ENOPOINTS 
A, 6 






COMMENCE 
SEARCH 
= (A + 8) /2 






/\ /s. 


FIGURE 9 
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with much better success (a copy of this routine can be 


mound at the end of the thesis). 


Bea RESULTS 
ime veCrimicacvomeo! Frrevious EStimates 

The majority of estimates in [5] Wiese Sveiqilisilercl las 
Pace computer Simulations. Table 1 lists the results for 
comparison. The values of uw are listed in displacement 
Peom Xo where X, is the smallest level at which a positive 
Pi Ons emocecUnrrccdn lis aiscplacement 15 given in multiples 
BreDELX (dx). The vatues of co are also given as multiples 
of DELX (dx). 

the .ecwainaves:. u and 3, for S; are not affected 
Mme tne total number of levels tested. However, the esti- 
meces trom an G5 Or 63 do depend on the total number of 
metals in the sequencer. The values for ~j and co for S; 
and S3 listed in Table 1 are valid for sequences of eight 
Seemore trials. If an investigator reaches S. or S3 with 
seven or fewer trials he should extend his search or repeat 
teralvogsethner. 

iiemirequency Of GCCUurrences Of S;, S>, and S$; 


from a sample of 2885 sequences was: 


frequency percentage 
S1 2226 (ORE 
S2 330 11.4 
S3 329 1a 4 


Ze 





pequerce Previous Estimates [5] Revised Estimates 


if 6 u S 
S} xt ax 2 iliee. 5 ex ee ~Oax fliers Gx 
So cee dx/4 6ax x +1. 2dx 3dax 
D3 x +13dx/4 bdx ~X<it1.8dx 3dax 


S; extension 


(three additional trials) 


outcome ft G u 6 

oF 0 0 xX t1.5dx i Dax Xt. 9dx TG 16358 
oo 1 Xt. (50x dx Xt. cdx dx 
ie 0 x,t dx Paix Xt. 3dx ieGidx 
eal 1 Xt. e5dx dx X 7 -edx dx 
mo 60 a dx 254 se X +. 30x Bae S8lo< 
i 1 = 2dax X— + 3dx ina eyelo 
imei 0 X, 25 DIS X—.3dx Ph ENG 
iL ae ae dx aye X--9dx dst 

ook = Dain. 
TABLE 1 


SIMULATION RESULTS 
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2. Biased Estimates 

Levels tested in the vicinity of the mean of the 
DMUs Glisurabution effer the most informative data, 
torre those tested Within a few standard deviations of 
the mean aid in pinning down the tails of the distribution. 
The Probit technique requires the investigation to com- 
mence fairly close to the mean. The Probit and Staircase 
methods use a constant step size between levels and thus 
Pea@rorm Desp when the step size = o.* For instance, if the 
ue sLZe >>o, then an investigator might never test within 
Peeee Or four Standard deviations of the mean. If the 
mee Size <<g, tnen Gne would need many trials to insure 
that an informative range of levels was tested in the 
vicinity of the mean. A useful estimate of o is one that 
fails within the range of o/2 to 2c. 

Miemvelivcowmatstedsan fable ivare functions of Xe 
and DELX, or DELX alone. The level Xo (smallest level 
at which a positive response occurs) is random but tends 
meward the mean of the distribution. DELX is also random 
muy depends upon the selection of the endpoints, a and b, 
the threshold variance, and the sequential outcomes 
Ms Yo ys. 5 yn): Thus DELX can take on values over a 
wide range implying tf and G can as well. 

On the basis of substantial sampling, estimates 
of i appear to be unbiased for S,, but seem to have a 
small predictable bias for S, and S3;. The average u from 


Simulated NQGt 1) Sensitivity data was about -0.1 for S, 
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Tiina roresane Hits WOuld correspond to py - .lo and p + .lo 
Bore N (i,0) data. 

The estimate G is on the other hand not so well 
behaved. It appears to have unpredictable bias for all 
sequences (Hicks Ree Less than half of the estimates 
of g generated from hundreds of simulated trials were within 
the desired range (c/2 to 20). The majority were less 
when o/2. 

Of primary concern is why the apparent bias? Is 
Equation (1) in error? Equation (1) does not depend upon 
Poieemumoer Of trials necessary for an inversion, whereas 
DELX does. It appears that MBS generates too many trials 
(i.e. DELX is frequently small in comparison to o when the 
first inversion occurs). Hence, the average estimate of 
¢ is much smaller than the true standard deviation. 

Consider an estimate 6 generated from an 8S, sequence 
Mane Howation (1). Table 1 lists ¢ = 1.3 DELX as the 
maximum likelihood estimate. An investigator would hope 
that if the true threshold variance were equal to 1.00, 

Maem DELA would be close to .77. If so, then G would 
appear unbiased and qualify as a useful input to Follow-up 
Mesto, However, the average length of DELX from 1000 S, 
sequences generated from N(0O, 1) sensitivity data equalled 
.49. With an S, or S3; sequence, an investigator would 

hope that DELX would be close to .33, since G = 3 DELX. 
Again a large sample (675) was investigated with an average 


PeeA OL .2l. Thus the maximum likelihood principal 


> 





(Equation (1)) yielded an average estimate of G that was 
PMbstancially omaller than the true standard deviation. 
forces 1Oethrough 12 are histograms of the frequency 
pcr onc Of DELX Tor S1, S2 and S3. Table e lists 
Boe Same data. 

A summary of the associated G's from the simulated 


sesequences.was as follows: 


GG 2 BE 6 & Blo 6 Bi 
S1 US 2% 50.6% N29 
Se Bong aaa 0.9% 
S3 51.1% 47 4% : ime 


The estimates of 6 were improved slightly by using 


an extension of three trials of S1. The resultS were as 
Hollows. 
Gao 2 RB 2 GS 26 G > 20 
YYZ 54% O% 


feeuMe 14 1 4 histogram of the latter distribution. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


Estimates of wu using MBS are suitable for follow-up 
tests while those of o are not. Additional trials will 
be necessary to refine the estimate of o. 

LomnigatLonworwmo With a small mumber of trials cannot 
Memeuaranteed. It is not possible for an investigator to 
gecermine beforehand the number of trials necessary to 
complete a terminal situation (S;, Sz, or S3). The average 
mumber generated in this investigation was eleven with 
extreme values of five and twenty-eight. It appears that 
miomnunmber 1s a function of the location of endpoints, 
the variance of the threshold distribution, and the sequen- 
tial outcomes of the experiment. 

The extension procedure in MBS should be investigated 
memenatelevels yielding the most useful data can be tested. 
For example, the middle level tested in an S: extension 
doesn't offer data to confirm or alter previous estimates 
of u and o whereas other levels may do so Significantly. 

Hoe an alternative approach to obtain practical estimates 
of o, a Monte Carlo method is suggested in place of the 
maximum likelihood principle. Note that sample values of 
DELX are "bunched" together in distinct intervals separated 
by intervals in which no value of DELX occurs (see Table 2). 
If an investigator could determine the interval to which 
the DELX from his experiment belongs, then he should be 


able to estimate o better than before. 
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